Perceptuo-affective organization of touched materials in younger and older adults.

In everyday interaction we touch different materials, which we experience along a limited number of perceptual and emotional dimensions: For instances, a furry surface feels soft and pleasant, whereas sandpaper feels rough and unpleasant. In a previous study, younger adults manually explored a representative set of solid, fluid and granular materials. Their ratings were made along six perceptual dimensions (roughness, fluidity, granularity, deformability, fibrousness, heaviness) and three emotional ones (valence, arousal, dominance). Perceptual and emotional dimensions were systematically correlated. Here, we wondered how this perceptuo-affective organization of touched materials depends on age, given that older adults show decline in haptic abilities, in particular detail perception. 30 younger participants (~22 years, half females) and 15 older participants (~66 years) explored 25 materials using 18 perceptual and 9 emotional adjectives. We extracted 6 perceptual and 2 emotional dimensions. Older and younger adults showed similar dimensions. However, in younger participants roughness and granularity judgments were done separately, while they were collapsed in a single dimension in older people. Further, age groups differed in the perception of roughness, granularity and valence, and older people did not show a positive correlation between valence and granularity as did younger people. As expected, control analyses between young males and females did not reveal similar gender differences. Overall, the results demonstrate that older people organize and experience materials partly differently from younger people, which we lead back to sensory decline. However, other aspects of perceptual organization that also include fine perception are preserved into older age.

Assessing the representational structure of softness activated by words.

Softness is an important material property that can be judged directly, by interacting with an object, but also indirectly, by simply looking at an image of a material. The latter is likely possible by filling in relevant multisensory information from prior experiences with soft materials. Such experiences are thought to lead to associations that make up our representations about perceptual softness. Here, we investigate the structure of this representational space when activated by words, and compare it to haptic and visual perceptual spaces that we obtained in earlier work. To this end, we performed an online study where people rated different sensory aspects of soft materials, presented as written names. We compared the results with the previous studies where identical ratings were made on the basis of visual and haptic information. Correlation and Procrustes analyses show that, overall, the representational spaces of verbally presented materials were similar to those obtained from haptic and visual experiments. However, a classifier analysis showed that verbal representations could better be predicted from those obtained from visual than from haptic experiments. In a second study we rule out that these larger discrepancies in representations between verbal and haptic conditions could be due to difficulties in material identification in haptic experiments. We discuss the results with respect to the recent idea that at perceived softness is a multidimensional construct.

A hierarchical sensorimotor control framework for human-in-the-loop robotic hands.

Human manual dexterity relies critically on touch. Robotic and prosthetic hands are much less dexterous and make little use of the many tactile sensors available. We propose a framework modeled on the hierarchical sensorimotor controllers of the nervous system to link sensing to action in human-in-the-loop, haptically enabled, artificial hands.

Simultaneous Emotional Stimuli Prolong the Timing of Vibrotactile Events.

Temporal information plays a crucial role in human everyday life. Yet, perceived time is subject to distortions. Emotion, for instance, is a powerful time modulator in that emotional events are perceived longer than neutral events of the same length. However, it is unknown how exposure to emotional stimuli influences the time perception of a simultaneous neutral tactile event. To fill this gap, we tested the effect of emotional auditory sounds on the perception of neutral vibrotactile feedback. We used neutral and emotional (i.e., pleasant-high arousal, pleasant-low arousal, unpleasant-high arousal, and unpleasant-low arousal) auditory stimuli from the International Digitized Sound System (IADS). Tactile information was a vibrotactile stimulus at a fixed intensity and presented through a custom-made vibrotactile sleeve. Participants listened to auditory stimuli which were temporally coupled with vibrotactile stimulation for 2,3,4, or 5 s. Their task was to focus on the duration of vibrotactile information and reproduce elapsed time. We tested the effects of valence and arousal of auditory stimuli on the perceived duration of vibrotactile information. Simultaneously presented auditory stimuli, in general, lengthened the perceived duration of the neutral vibrotactile information compared to neutral auditory stimuli. We conclude that emotional events influence time perception of simultaneous neutral haptic events.

The role of spatial information in an approximate cross-modal number matching task.

The approximate number system (ANS) is thought to be an innate cognitive system that allows humans to perceive numbers (>4) in a fuzzy manner. One assumption of the ANS is that numerosity is represented amodally due to a mechanism, which filters out nonnumerical information from stimulus material. However, some studies show that nonnumerical information (e.g., spatial parameters) influence the numerosity percept as well. Here, we investigated whether there is a cross-modal transfer of spatial information between the haptic and visual modality in an approximate cross-modal number matching task. We presented different arrays of dowels (haptic stimuli) to 50 undergraduates and asked them to compare haptically perceived numerosity to two visually presented dot arrays. Participants chose which visually presented array matched the numerosity of the haptic stimulus. The distractor varied in number and displayed a random pattern, whereas the matching (target) dot array was either spatially identical or spatially randomized (to the haptic stimulus). We hypothesized that if a "numerosity" percept is based solely on number, neither spatially identical nor spatial congruence between the haptic and the visual target arrays would affect the accuracy in the task. However, results show significant processing advantages for targets with spatially identical patterns and, furthermore, that spatial congruency between haptic source and visual target facilitates performance. Our results show that spatial information was extracted from the haptic stimuli and influenced participants' responses, which challenges the assumption that numerosity is represented in a truly abstract manner by filtering out any other stimulus features.

Hand explorations are determined by the characteristics of the perceptual space of real-world materials from silk to sand.

Perceiving mechanical properties of objects, i.e., how they react to physical forces, is a crucial ability in many aspects of life, from choosing an avocado to picking your clothes. There is, a wide variety of materials that differ substantially in their mechanical properties. For example, both, silk and sand deform and change shape in response to exploration forces, but each does so in very different ways. Studies show that the haptic perceptual space has multiple dimensions corresponding to the physical properties of textures, however in these experiments the range of materials or exploratory movements were restricted. Here we investigate the perceptual dimensionality in a large set of real materials in a free haptic exploration task. Thirty-two participants actively explored deformable and non-deformable materials with their hands and rated them on several attributes. Using the semantic differential technique, video analysis and linear classification, we found four haptic dimensions, each associated with a distinct set of hand and finger movements during active exploration. Taken together our findings suggest that the physical, particularly the mechanical, properties of a material systematically affect how it is explored on a much more fine-grained level than originally thought.

Tactile suppression stems from specific sensorimotor predictions.

The ability to sample sensory information with our hands is crucial for smooth and efficient interactions with the world. Despite this important role of touch, tactile sensations on a moving hand are perceived weaker than when presented on the same but stationary hand. This phenomenon of tactile suppression has been explained by predictive mechanisms, such as internal forward models, that estimate future sensory states of the body on the basis of the motor command and suppress the associated predicted sensory feedback. The origins of tactile suppression have sparked a lot of debate, with contemporary accounts claiming that suppression is independent of sensorimotor predictions and is instead due to an unspecific mechanism. Here, we target this debate and provide evidence for specific tactile suppression due to precise sensorimotor predictions. Participants stroked with their finger over textured objects that caused predictable vibrotactile feedback signals on that finger. Shortly before touching the texture, we probed tactile suppression by applying external vibrotactile probes on the moving finger that either matched or mismatched the frequency generated by the stroking movement along the texture. We found stronger suppression of the probes that matched the predicted sensory feedback. These results show that tactile suppression is specifically tuned to the predicted sensory states of a movement.

Get in touch with numbers - an approximate number comparison task in the haptic modality.

The Approximate Number System (ANS) is conceptualized as an innate cognitive system that allows humans to perceive numbers of objects or events (>4) in a fuzzy, imprecise manner. The representation of numbers is assumed to be abstract and not bound to a particular sense. In the present study, we test the assumption of a shared cross-sensory system. We investigated approximate number processing in the haptic modality and compared performance to that of the visual modality. We used a dot comparison task (DCT), in which participants compare two dot arrays and decide which one contains more dots. In the haptic DCT, 67 participants had to compare two simultaneously presented dot arrays with the palms of their hands; in the visual DCT, participants inspected and compared dot arrays on a screen. Tested ratios ranged from 2.0 (larger/smaller number) to 1.1. As expected, in both the haptic and the visual DCT responses similarly depended on the ratio of the numbers of dots in the two arrays. However, on an individual level, we found evidence against medium or stronger positive correlations between "ANS acuity" in the visual and haptic DCTs. A regression model furthermore revealed that besides number, spacing-related features of dot patterns (e.g., the pattern's convex hull) contribute to the percept of numerosity in both modalities. Our results contradict the strong theory of the ANS solely processing number and being independent of a modality. According to our regression and response prediction model, our results rather point towards a modality-specific integration of number and number-related features.

Different contributions of efferent and reafferent feedback to sensorimotor temporal recalibration.

Adaptation to delays between actions and sensory feedback is important for efficiently interacting with our environment. Adaptation may rely on predictions of action-feedback pairing (motor-sensory component), or predictions of tactile-proprioceptive sensation from the action and sensory feedback of the action (inter-sensory component). Reliability of temporal information might differ across sensory feedback modalities (e.g. auditory or visual), which in turn influences adaptation. Here, we investigated the role of motor-sensory and inter-sensory components on sensorimotor temporal recalibration for motor-auditory (button press-tone) and motor-visual (button press-Gabor patch) events. In the adaptation phase of the experiment, action-feedback pairs were presented with systematic temporal delays (0 ms or 150 ms). In the subsequent test phase, audio/visual feedback of the action were presented with variable delays. The participants were then asked whether they detected a delay. To disentangle motor-sensory from inter-sensory component, we varied movements (active button press or passive depression of button) at adaptation and test. Our results suggest that motor-auditory recalibration is mainly driven by the motor-sensory component, whereas motor-visual recalibration is mainly driven by the inter-sensory component. Recalibration transferred from vision to audition, but not from audition to vision. These results indicate that motor-sensory and inter-sensory components contribute to recalibration in a modality-dependent manner.

The look and feel of soft are similar across different softness dimensions.

The softness of objects can be perceived through several senses. For instance, to judge the softness of a cat's fur, we do not only look at it, we often also run our fingers through its coat. Recently, we have shown that haptically perceived softness covaries with the compliance, viscosity, granularity, and furriness of materials (Dovencioglu, Üstün, Doerschner, & Drewing, 2020). However, it is unknown whether vision can provide similar information about the various aspects of perceived softness. Here, we investigated this question in an experiment with three conditions: in the haptic condition, blindfolded participants explored materials with their hands, in the static visual condition participants were presented with close-up photographs of the same materials, and in the dynamic visual condition participants watched videos of the hand-material interactions that were recorded in the haptic condition. After haptically or visually exploring the materials, participants rated them on various attributes. Our results show a high overall perceptual correspondence among the three experimental conditions. With a few exceptions, this correspondence tended to be strongest between haptic and dynamic visual conditions. These results are discussed with respect to information potentially available through the senses, or through prior experience, when judging the softness of materials.

Target Search and Inspection Strategies in Haptic Search.

Haptic search is a common everyday task, usually consisting of two processes: target search and target analysis. During target search we need to know where our fingers are in space, remember the already completed path and the outline of the remaining space. During target analysis we need to understand whether the detected potential target is the desired one. Here we characterized dynamics of exploratory movements in these two processes. In our experiments participants searched for a particular configuration of symbols on a rectangular tactile display. We observed that participants preferentially moved the hand parallel to the edges of the tactile display during target search, which possibly eased orientation within the search space. After a potential target was detected by any of the fingers, there was higher probability that subsequent exploration was performed by the index or the middle finger. At the same time, these fingers dramatically slowed down. Being in contact with the potential target, the index and the middle finger moved within a smaller area than the other fingers, which rather seemed to move away to leave them space. These results suggest that the middle and the index finger are specialized for fine analysis in haptic search.

Task and Material Properties Interactively Affect Softness Explorations Along Different Dimensions.

Haptic research has frequently equated softness with the compliance of elastic objects. However, in a recent study we have suggested that compliance is not the only perceived material dimension underlying what is commonly called softness [1 ]. Here, we investigate, whether the different perceptual dimensions of softness affect how materials are haptically explored. Specifically, we tested whether also the task, i.e., the attribute that a material is being judged on, might affect how a material is explored. To this end we selected 15 adjectives and 19 materials that each associate with different softness dimensions for the study. In the experiment, while participants freely explored and rated the materials, we recorded their hand movements. These movements were subsequently categorized into distinct exploratory procedures (EPs) and analyzed in a multivariate analysis of variance (MANOVA). The results of this analysis suggest that the pattern of EPs depended not only on the material's softness dimension and the task (i.e., what attributes were rated), but also on an interaction between the two factors. Taken together, our findings support the notion of multiple perceptual dimensions of softness and suggest that participants actively adapt their EPs in a nuanced way when judging a particular softness dimensions for a given material.

Masking interferes with haptic texture perception from sequential exploratory movements.

Haptic texture perception is based on sensory information sequentially gathered during several lateral movements ("strokes"). In this process, sensory information of earlier strokes must be preserved in a memory system. We investigated whether this system may be a haptic sensory memory. In the first experiment, participants performed three strokes across each of two textures in a frequency discrimination task. Between the strokes over the first texture, participants explored an intermediate area, which presented either a mask (high-energy tactile pattern) or minimal stimulation (low-energy smooth surface). Perceptual precision was significantly lower with the mask compared with a three-strokes control condition without an intermediate area, approaching performance in a one-stroke-control condition. In contrast, precision in the minimal stimulation condition was significantly better than in the one-stroke control condition and similar to the three-strokes control condition. In a second experiment, we varied the number of strokes across the first stimulus (one, three, five, or seven strokes) and either presented no masking or repeated masking after each stroke. Again, masking between the strokes decreased perceptual precision relative to the control conditions without masking. Precision effects of masking over different numbers of strokes were fit by a proven model on haptic serial integration (Lezkan & Drewing, Attention, Perception, & Psychophysics 80(1): 177-192, 2018b) that modeled masking by repeated disturbances in the ongoing integration. Taken together, results suggest that masking impedes the processes of haptic information preservation and integration. We conclude that a haptic sensory memory, which is comparable to iconic memory in vision, is used for integrating sequentially gathered sensory information.

Deep neural network model of haptic saliency.

Haptic exploration usually involves stereotypical systematic movements that are adapted to the task. Here we tested whether exploration movements are also driven by physical stimulus features. We designed haptic stimuli, whose surface relief varied locally in spatial frequency, height, orientation, and anisotropy. In Experiment 1, participants subsequently explored two stimuli in order to decide whether they were same or different. We trained a variational autoencoder to predict the spatial distribution of touch duration from the surface relief of the haptic stimuli. The model successfully predicted where participants touched the stimuli. It could also predict participants' touch distribution from the stimulus' surface relief when tested with two new groups of participants, who performed a different task (Exp. 2) or explored different stimuli (Exp. 3). We further generated a large number of virtual surface reliefs (uniformly expressing a certain combination of features) and correlated the model's responses with stimulus properties to understand the model's preferences in order to infer which stimulus features were preferentially touched by participants. Our results indicate that haptic exploratory behavior is to some extent driven by the physical features of the stimuli, with e.g. edge-like structures, vertical and horizontal patterns, and rough regions being explored in more detail.

A Systematic Comparison of Perceptual Performance in Softness Discrimination with Different Fingers.

In studies investigating haptic softness perception, participants are typically instructed to explore soft objects by indenting them with their index finger. In contrast, performance with other fingers has rarely been investigated. We wondered which fingers are used in spontaneous exploration and if performance differences between fingers can explain spontaneous usage. In Experiment 1 participants discriminated the softness of two rubber stimuli with hardly any constraints on finger movements. Results indicate that humans use successive phases of different fingers and finger combinations during an exploration, preferring index, middle, and (to a lesser extent) ring finger. In Experiment 2 we compared discrimination thresholds between conditions, with participants using one of the four fingers of the dominant hand. Participants compared the softness of rubber stimuli in a two-interval forced choice discrimination task. Performance with index and middle finger was better as compared to ring and little finger, the little finger was the worst. In Experiment 3 we again compared discrimination thresholds, but participants were told to use constant peak force. Performance with the little finger was worst, whereas performance for the other fingers did not differ. We conclude that in spontaneous exploration the preference of combinations of index, middle, and partly ring finger seems to be well chosen, as indicated by improved performance with the spontaneously used fingers. Better performance seems to be based on both different motor abilities to produce force, mainly linked to using index and middle finger, and different sensory sensitivities, mainly linked to avoiding the little finger.

The size-weight illusion comes along with improved weight discrimination.

When people judge the weight of two objects of equal mass but different size, they perceive the smaller one as being heavier. Up to date, there is no consensus about the mechanisms which give rise to this size-weight illusion. We recently suggested a model that describes heaviness perception as a weighted average of two sensory heaviness estimates with correlated noise: one estimate derived from mass, the other one derived from density. The density estimate is first derived from mass and size, but at the final perceptual level, perceived heaviness is biased by an object's density, not by its size. Here, we tested the models' prediction that weight discrimination of equal-size objects is better in lifting conditions which are prone to the size-weight illusion as compared to conditions lacking (the essentially uninformative) size information. This is predicted because in these objects density covaries with mass, and according to the model density serves as an additional sensory cue. Participants performed a two-interval forced-choice weight discrimination task. We manipulated the quality of either haptic (Experiment 1) or visual (Experiment 2) size information and measured just-noticeable differences (JNDs). Both for the haptic and the visual illusion, JNDs were lower in lifting conditions in which size information was available. Thus, when heaviness perception can be influenced by an object's density, it is more reliable. This discrimination benefit under conditions that provide the additional information that objects are of equal size is further support for the role of density and the integration of sensory estimates in the size-weight illusion.

Memory influences haptic perception of softness.

The memory of an object's property (e.g. its typical colour) can affect its visual perception. We investigated whether memory of the softness of every-day objects influences their haptic perception. We produced bipartite silicone rubber stimuli: one half of the stimuli was covered with a layer of an object (sponge, wood, tennis ball, foam ball); the other half was uncovered silicone. Participants were not aware of the partition. They first used their bare finger to stroke laterally over the covering layer to recognize the well-known object and then indented the other half of the stimulus with a probe to compare its softness to that of an uncovered silicone stimulus. Across four experiments with different methods we showed that silicon stimuli covered with a layer of rather hard objects (tennis ball and wood) were perceived harder than the same silicon stimuli when being covered with a layer of rather soft objects (sponge and foam ball), indicating that haptic perception of softness is affected by memory.

Neural correlates of top-down modulation of haptic shape versus roughness perception.

Exploring an object's shape by touch also renders information about its surface roughness. It has been suggested that shape and roughness are processed distinctly in the brain, a result based on comparing brain activation when exploring objects that differed in one of these features. To investigate the neural mechanisms of top-down control on haptic perception of shape and roughness, we presented the same multidimensional objects but varied the relevance of each feature. Specifically, participants explored two objects that varied in shape (oblongness of cuboids) and surface roughness. They either had to compare the shape or the roughness in an alternative-forced-choice-task. Moreover, we examined whether the activation strength of the identified brain regions as measured by functional magnetic resonance imaging (fMRI) can predict the behavioral performance in the haptic discrimination task. We observed a widespread network of activation for shape and roughness perception comprising bilateral precentral and postcentral gyrus, cerebellum, and insula. Task-relevance of the object's shape increased activation in the right supramarginal gyrus (SMG/BA 40) and the right precentral gyrus (PreCG/BA 44) suggesting that activation in these areas does not merely reflect stimulus-driven processes, such as exploring shape, but also entails top-down controlled processes driven by task-relevance. Moreover, the strength of the SMG/PreCG activation predicted individual performance in the shape but not in the roughness discrimination task. No activation was found for the reversed contrast (roughness > shape). We conclude that macrogeometric properties, such as shape, can be modulated by top-down mechanisms whereas roughness, a microgeometric feature, seems to be processed automatically.

Integration of prior knowledge during haptic exploration depends on information type.

When haptically exploring softness, humans use higher peak forces when indenting harder versus softer objects. Here, we investigated the influence of different channels and types of prior knowledge on initial peak forces. Participants explored two stimuli (hard vs. soft) and judged which was softer. In Experiment 1 participants received either semantic (the words "hard" and "soft"), visual (video of indentation), or prior information from recurring presentation (blocks of harder or softer pairs only). In a control condition no prior information was given (randomized presentation). In the recurring condition participants used higher initial forces when exploring harder stimuli. No effects were found in control and semantic conditions. With visual prior information, participants used less force for harder objects. We speculate that these findings reflect differences between implicit knowledge induced by recurring presentation and explicit knowledge induced by visual and semantic information. To test this hypothesis, we investigated whether explicit prior information interferes with implicit information in Experiment 2. Two groups of participants discriminated softness of harder or softer stimuli in two conditions (blocked and randomized). The interference group received additional explicit information during the blocked condition; the implicit-only group did not. Implicit prior information was only used for force adaptation when no additional explicit information was given, whereas explicit interfered with movement adaptation. The integration of prior knowledge only seems possible when implicit prior knowledge is induced-not with explicit knowledge.

Effects of Stimulus Exploration Length and Time on the Integration of Information in Haptic Softness Discrimination.

In haptic perception, information is often sampled serially (e.g., a stimulus is repeatedly indented to estimate its softness), requiring that sensory information is retained and integrated over time. Hence, integration of sequential information is likely affected by memory. Particularly, when two sequentially explored stimuli are compared, integration of information on the second stimulus might be determined by the fading representation of the first stimulus. We investigated how the exploration length of the first stimulus and a temporal delay affect contributions of sequentially gathered estimates of the second stimulus in haptic softness discrimination. Participants subsequently explored two silicon rubber stimuli by indenting the first stimulus one or five times and the second stimulus always three times. In an additional experiment, we introduced a 5-s delay after the first stimulus was indented five times. We show that the longer the first stimulus is explored, the more estimates of the second stimulus' softness contribute to the discrimination of the two stimuli, independent of the delay. This suggests that the exploration length of the first stimulus influences the strength of its representation, persisting at least for 5 s, and determines how much information about the second stimulus is exploited for the comparison.